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Abstract: Key to renewed concern on the affective domain of education (Fensham, 2007) and on school 
graduates’ readiness for a world of work (BEST, 2008; WDA, 2006) is the student’s inclination-to-reflect when 
engaged in a learning or prohlem-solving task. Reflective learning is not new to education (Dewey, 1933; Ellis, 
2001). Since the inclination-to-reflect may not he obvious even among adults at work (Seihert & Daudelin, 1999), 
how much more can we expect from school students? This article presents part of a research on secondary school 
students’ inclination-to-reflect when engaged in chemistry learning tasks. The instrument used is the three-part 
Chemistry Eearning and Thinking Instrument (CETI). The first part seeks to characterize students’ 
inclination-to-reflect while attempting chemistry learning tasks and the other parts aim to characterize their 
learning-thinking preferences in the subject. This article shares the construction of the learning tasks in the first 
part and how students’ reflective responses to these tasks are encouraged, scored and analyzed. Since assessment 
is said to drive teaching and learning, an alternative form of assessment, such as these CETI items, may help 
students become more reflective in their learning habits and hence more adaptable to the world of work. 

Key words: affective science education; Chemistry Eearning and Thinking Instrument (CETI); 
inclination-to-reflect; reflective learning 

1. Introduction 

Renewed concerns on the affective domain of education (Eensham, 2007) and on school graduates’ readiness 
to enter the world of work (Department of Education, Employment and Workplace Relations, BEST, 2008; Work 
Development Authority, WDA, 2006) point to an important development in the way students are being prepared in 
school — their approaches to solving problems. Being reflective is a well-known thinking habit among scientists. 
Good examples include renowned scientists like Isaac Newton (1643-1727, discoverer of gravity), Gregor Mendel 
(1822-1884, the father of hereditary science) and August Kekule (1829-1896, who first proposed the molecular 
structure of benzene in 1865). Being reflective is also encouraged and recognised as an essential habit among 
good problem solvers (Stroulia & Goel, 1994) and in everyday life (Brockbank, McGill & Beech, 2002; Covey, 
1989; Harrison, 2006). Since assessment is often said to drive learning habits (Enger & Yager, 2001; Gronlund & 
Waugh, 2008), it may therefore be useful to consider assessing students’ inclination-to-reflect so that students can 
be encouraged to become habitual reflective learners. Hopefully, this will also better prepare them for a more 
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complex and demanding world of work. 

This article presents a part of a research on secondary school students’ inclination-to-reflect while engaged in 
Chemistry learning tasks. The instrument used is the three -part Chemistry Learning and Thinking Instrument, or 
CLTI. The first part seeks to characterize students’ inclination-to-reflect while attempting Chemistry learning tasks 
and the other two parts aim to characterize their learning-thinking preferences in the subject. Only the first part of 
the CLTI will he discussed, including how students’ reflective responses to the learning tasks are scored and 
compared. The purpose is not to report in detail the findings of the research hut to share an alternative way to 
assess students’ ability to solve problems or respond to learning tasks in school chemistry. 

2. Defining reflective learning 

The reflective approach to learning is not a new idea in education. Among the earliest recorded works 
relating to reflective learning are those of Socrates (the great 5th Century B.C. Greek philosopher). Socrates once 
said, “The unexamined life is not worth living” (Brickhouse & Smith, 1996, p. 209). Though this may sound harsh, 
it is undeniable that the human race would not have made the progress we see now if people had not been 
observant of their surrounding, asked the appropriate questions and generated possibilities to work on so that our 
lives can be more comfortable. In 1933, renowned educationist, John Dewey had written about reflective thinking 
and the education of man. Dewey’s work suggested that to understand life experiences better, we can “... take 
stock of the conditions before suggestions arise of possible course of action” (Dewey, 1933, pp. 102-103). This 
involves the learner taking time to pause and look into the future, recapturing past experiences and establishing a 
relationship between these experiences and thoughts on a new basis. The experiential learning cycle was then 
proposed by Kolb (1984) as a model of learning with reflection as a key element. Kolb’s model explained how 
reflecting on experiences helps to further develop the person’s learning capacity. 

The literature may be extensive and reaches far back in history but few are empirical studies about the 
intervening applications of reflective learning strategies. Most of the reported work deal with adult learning and 
professional practices (Branch & Paranjape, 2002; Loughran, 1996; Moon, 2004; Pollard, 2002; Schon, 1983, 
1987; Seibert & Daudeline, 1999; Tan & Ee, 2004; Taylor, 2006). Those that relate to younger learners and school 
students appear to be reports on best practices or sharing of a variety of learning-teaching strategies (Trudeau & 
Harle, 2006; Wilson & Jan, 1993). However, in almost all reports two important precursors to reflection — time 
and experience, were frequently reported. Reflective learning may then be defined or described as a form of 
learning that requires the learner to pause and observe her or his learning situation by considering past relevant 
experiences and using these to generate useful information from the present situation. The learner can then make 
sense of the learning process by linking the past, the present and/or the future. 

3. Applying reflective learning 

Many reflective learning strategies have been suggested or described as applicable in a classroom or science 
laboratory setting (Ellis, 2001; Eogarty, 1994; Tan, 2002, 2005, 2007; Whitaker, 1995; Wilson & Jan, 1993). 
Practising classroom teachers are probably familiar with procedural strategies like: (1) KWE, which is commonly 
used in a lesson to identify “what I Know, Want of know and what I have Eearnt”; (2) Question authoring, like the 
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use of “What if . . questions, during a lesson or project involving critical thinking and prohlem solving (Channel 
News Network, CNN, 2003; Fogarty, 1994; Tan, 2007), and review procedures like (3) Establishing clear-unclear 
windows (Ellis, 2001; Eogarty, 1994), which may include activities like concept mapping and learning logs 
written as a form of self-evaluation before, during and after a lesson. These strategies require learners to reflect. 
Many similar strategies are also applicable in experimental science or laboratory practical activities. Eor example, 
inquiry -related strategies like (1) POE (Predict-Observe-Explain) are commonly used before or during an 
experimental lesson and (2) recovery strategies (Ellis, 2001; Eogarty, 1994), for trouble-shooting during problem 
solving in a practical session. Since teachers are already employing some forms of reflective learning strategies in 
their lessons, it is therefore reasonable to state that reflective learning is not a new idea in education. 

Reflective learning strategies are also metacognitive in nature (Eogarty, 1994), that is, students are 
encouraged to think about their thinking (Elavell, 1976, 1979). Exposing them to such learning strategies may 
develop them into metacognitive and reflective learners. Being reflective means the learner not only has to think 
about her or his own learning-thinking approaches (metacognitive) but also to make sense of what they have done, 
are doing and will proceed to do. In other words, they become actively engaged in what Schon (1983) called in his 
book. The Reflective Practitioner, “reflection-in-action” (reflection while the event is still going on) and 
“reflection-on-action” (reflection at the end of an event). 

Although it appears simple to describe reflection as a learning habit or strategy, it may not be as easy to 
conclude if someone is reflecting or not. Reflecting as a human activity is a form of thinking that that cannot be 
visibly observed. Nobody can observe another person’s thinking process visually. The same may be said of 
reflection. It is therefore necessary to provide “proxies” to indicate that a learner is, or has indeed been, reflecting 
and not daydreaming or pretending to think. Hence, the survey instrument used in the present research provides 
opportunities to collect some concrete evidences on student’s reflection. 

4. Students’ inclination-to-reflect in learning school chemistry 

Based on the reflection literature, a model was established by Tan and Goh (2002) to explain how reflective 
learning could take place in the classroom. A strategy that may be applied in the classroom learning situation 
comprises three skills commonly illustrated in most reflective learning activities. These are namely, observe, 
generate and relate (Eigure 1). The reflective learning model and strategy were further developed by Tan (2008) to 
include the roles of reflective teaching in a classroom lesson (Eigure 2). This model was then used in the 
construction of an instrument to characterize the reflective learning-thinking approaches of secondary three 
Chemistry students in Singapore. The instrument, known as the Chemistry Eeaming and Thinking Instrument (or 
CETI), includes a list of ten chemistry-related learning tasks. Students’ scores on these tasks were compared to the 
sample’s mean and used to place each student in the “high” group (more “inclined-to-reflect” compared to the 
sample’s mean performance) or the “low” group (less “inclined-to-reflect”). 
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Figure 1 




The OGR reflective learning strategy in the classroom 




Figure 2 Model to explain reflective teaching and reflective learning in the classroom (Tan, 2008) 



The CLTI is not a psychometric instrument used for measuring the degree of reflectiveness in learning. 
Instead, it is used to characterize students’ learning-thinking approaches. The items serve to engage students in 
generating alternative and meaningful solutions or responses to the Chemistry learning tasks. Students are 
required to respond to all items hy writing down their answers in the spaces provided. There are no structured 
prompts to help students respond. That is, no boxes or brackets for ticking and no lines are provided for writing 
because students may take these as indications of how many possible answers or alternatives they can generate. 
They are only told to write their responses in the blank spaces provided. All questions (with a few exceptions) 
carry the general instruction to “list all possible answers or alternative solutions” to the task. This is to engage the 
more reflective students to take time to think and link their own experiences with the learning tasks and generate 
more relevant alternatives or possible solutions. A two-tier scoring scheme is then used to evaluate the students’ 
responses (Table 1). 



Table 1 Two-tier scoring scheme for students’ responses to the items in the CLTI 



Tier 


If the student is to make a 


Score 


1 


relevant or related response 


one point 


2 


correct or accurate response 


a maximum of two points 



A relevant or related response may be completely or partially correct. A correct or accurate response* refers to 
a conceptually correct response to the learning task. The total number of relevant or related responses will then 
become the student’s first tier score. The scorer will then re-visit the list of responses, evaluates the correctness or 



* All items and responses are subjected to vetting by a panel of experts. The inter-scorer consistency for the items was at 0.75. 
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accuracy of each response and awards the student the second tier scores for the same responses. The overall 
performance for that student on a particular learning task will he the sum of the two scores. Thus, the higher the 
student’s overall score, the more inclined she or he is to reflect. 

As an example of how a student’s response was scored, consider a student who had listed three correct 
alternatives or possibilities. She or he would have scored [(3xl)-i-(3x2)], that is 9 points. If out of the three 
responses related to the task, only two were correct and the remaining was either incorrect or inaccurate, then she 
or he would have scored [(3xl)-i-(2x2)-i-(lx0)], that is 7 points. Similarly, if all three responses were incorrect hut 
relevant to the theme of the task, then the score would he [(3xl)-i-(3x0)], that is 3 points. Figure 3 shows at typical 
CLTI item and Figure 4 presents examples of some actual students’ responses for this item. 



The following diagram shows a chromatogram obtained using 
soiutions of three singie dyes (blue, green and red) and four other 
solutions (K, L, M and N) that contain one or more dyes. 
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List all the possible conclusions that can be made from observing the 
chromatogram obtained above. 

Figure 3 Sample item of the CLTI (Topic on chromatography, Item B9) 



Example A: All generated conclusions are relevant, connect and accurate (Respondent MD025). 

< 15 /\ 

L fs Ov puff 

N rs A (Ay< ■ 

Scoring: All four points are relevant and correct. Hence the score for this response is [(4 x 1) + (4 x 2)] = 12 points. 

Example B: Same task but with one incorrect conclusion (Respondent MDOlO). 

< is wi«v)lei/ip of feddije i\v,cI-nAOiAVitviO(^Jocfcj«' ^ owcte ypaf .4 o .... 

3’wir\ciuiP (xnd rPddj-f 

L AS fnowiewp 0^ tlvie 

M is 0 C hi ritioo(/4ki . 0Vcevi dyp 

Scoring: All five conclusions are relevant, but only the first conclusion is incon'ect. Hence the score for this response is [(5 x 
1) -h (4 X 2) -h (1 X 0)] = 13 points. 
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(continued) 



Example C: Partially correct conclusions (Respondent HB003). 




K covtftim d\jf( ^ And 




L ocn-f^lrij bivte ' 


1C , (tnd f/) 










Scoring: All eight conclusions are relevant to the task requirement but the fifth conclusion is partially correct (M does not 
contain the red dye). The sixth and seventh points are repeats of observations made in the earlier points but these are correct 
conclusions. On a second tier scoring, these three conclusions may be awarded one point (out of two) each. Hence the score 
for this response is [(8xl)+(5x2)+(3xl)]=21 points. 



Figure 4 Examples on scoring responses to CLTI item 



In example C (Figure 4), the scorer had also awarded 1 point out of 2 possible points to a partially correct or 
accurate response. Hence, the scoring on the second tier allows some leeway for the scorer to gauge the quality of 
a relevant response. 

The student’s total score for the ten items were then computed and compared to the mean total score of the 
sample. Students who scored higher than the sample’s mean total score would he identified with an “H” 
characteristic for being more inclined-to-reflect compared to the rest. Those with a score lower than the sample’s 
mean total score fall within the group identified with an “L” characteristic. 

5. Results and findings 

The CLTI was administered as part of a doctoral research on Students’ Ideas in Designing Experimental 
Set-Ups and their Reflective Learning-Thinking Approaches in School Chemistry (Tan, 2008). The sample studied 
for this report comprises 124 secondary school chemistry students who had completed at least one year of study at 
upper secondary level (at the end of secondary three or early secondary four). Figure 5 shows the distribution and 
ranking of the students’ scores for the ten-item CLTI survey. 



Ranking of students' scores on reflective assessment items 




student 



Mean 


1%21 


Median 


78.00 


Mode 


65.00 


Range 


105.00 


Minimum 


27.00 


Maximum 


132.00 


SD 


2.25 



Figure 5 Distribution and ranking of students’ scores for the ten-item CLTI survey 



The results were used in the original study to answer the research question on whether students who are more 
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inclined-to-reflect can design better chemistry experimental set-ups. “Good designers” in the study are those who 
scored consistently well in designing tasks listed in a separate four-task survey form, the Experimental Design 
Task List (EDT List). These “good designers” were frequently found to have higher scores on inclination-to 
-reflect in the CLTI survey (Tan, 2008). The group with lower scores were found to perform poorly on the EDT 
List. Hence, the study provided some empirical evidence to show that being reflective in their habit of learning 
may have helped students perform better in problem-solving tasks like designing experimental set-ups. 

6. Discussion and recommendations 

The main research using the survey instruments (CLTI and EDT List) has yielded some useful findings, 
including evidences that “good designers” of experimental set-ups are more inclined-to-reflect compared to the 
“poorer designers”. The more reflective students could have been more observant and thus identified more 
possibilities or alternatives. These could have helped them design more appropriate set-ups for the experimental 
design tasks presented in the survey. 

Two of the several recommendations made in the study are: (1) to re-word assessment tasks commonly found 
in most examination papers in an open-ended manner in order to encourage reflective responses from students, 
and (2) to assess students periodically using reflective assessment items (like those in the CLTI). Using the 
two-tier mark scheme, or similarly crafted mark schemes, it is possible to grade students over a range of scores, 
with the higher scores being indicative of students’ greater inclination-to-reflect. Students may become habitual 
reflective learners if they realize that being reflective in their learning approaches can help them solve problems 
more effectively. 

There are limitations to a reflective mode of assessment, like: 

(1) A large amount of time has to be spent crafting reflective assessment items or re-crafting from typical 
assessment items, and in scoring students’ reflective responses; 

(2) Ambiguities in students’ reflective responses, since these assessment items are open-ended and there can 
be a host of possible responses and; 

(3) The readiness of teachers to embrace unfamiliar situations posed by the open-ended nature of the 
reflective assessment items and the less predictable responses from their students. 

The limitations are important and should therefore be carefully considered in future efforts relating to 
assessment of students based on their inclination-to-reflect. One strategy to tackle the problem of having to spend 
a lot of time in teaching and preparing students to respond to reflective assessment items is to frequently expose 
students to short reflective learning exercises. Also, including one or two reflective assessment items in a typical 
examination could encourage students to adopt a more reflective approach to their learning. Ambiguities 
surrounding some of the students’ responses may also be dealt with during professional discussions and 
documented by teachers and examiners so that experiences learnt may be used to craft similar items in future 
assessment. The research also found that establishing an inter-scorer consistency among randomly selected items 
helped make the reflective assessment items more reliable. Linally, teachers preparing students for reflective 
assessment need to be proficient both in content knowledge and in pedagogical skills. Continual professional 
development for teachers in guiding students towards a more reflective mode of learning in class is therefore 
recommended. 
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7. Conclusion 

Research on developing objective ways to measure reflectiveness in learning had previously led to critical 
discussions (Kagan, 1965; Krumboltz, 1965). Despite the lack of empirical studies on reflective learning, 
especially within the school context, it is obvious that measuring reflectiveness in learning can be difficult, if not 
impossible. For example, it can be doubtful to suggest that learner A is twice or three times more reflective than 
learner B. The scores and analyses from the CLTI only serve to describe students as being more or less 
inclined-to-reflect when compared to the sample’s mean. Although this may appear to be a straightforward 
research strategy, it is probably a first step towards assessing students on their ability to solve problems through a 
reflective approach. This different mode of assessment may help students change their strategy in preparing for 
assessment. Instead of blindly practising to solve problems and mugging to remember facts so as to achieve high 
test scores, they may now strive to generate alternative relevant ideas to a learning task or problem, thus engaging 
them to reflect critically and frequently. Despite the limitations and apparent straightforwardness in the conduct of 
the study, the results do hold some promises that a more objective way to assess students’ reflective responses may 
become part of typical school examinations in future. It will be worth the time and effort to conduct a reflective 
mode of assessment since it can increase the chances of nurturing the younger generation of learners to become 
more reflective. In a fast-paced world fuelled by an expanding knowledge base and supported by highly 
technology-dependent systems, being less impulsive and making informed decisions may be critical soft skills 
students will need now and in the near future. To help students acquire such skills, one way forward is to 
encourage reflective learning in schools. 
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